Firewalls outbound application filtering VS Leak Tests

WIN 2000/XP

Highest Setting (last global update : October 13 2004)
Firewall AM ver(build) LeakTest ToolLeaky FireHole Yalta PCAudit AWFTester Thermite CopyCat MBtest WBreaker PCAudit2 Ghost DNStester Surfer *Score*

Zone Alarm
5.5.035 beta 7/10 17/24

Kerio
4.1.1 1/10 5/24

Outpost
2.5(369/369) 10/10 18/24

Look'n'Stop
2.05p2 10/10 19/24

Norton
2005 (8.0.0.64) 1/10 7/24

Sygate
5.5(2637) 2/10 9/24

Jetico
1.0.1.21 beta 8/10 16/24

Kaspersky
1.5.119.0 1/10 3/24

SP1
- 0/10 0/24

SP2
- 0/10 0/24
Click Here to see the tables legend
Please read the NEWS N°44 annoucing the update of these tests and giving more information.

These results does NOT show the ranking of the overall firewall features, but only of a single component, the outbound application filtering. If you want to know how to do the tests yourself or if you want to understand the results, read the PDF document below

=> results details/explanation Here

Out of the box

Due to the increase number of firewalls to test, I did not include this time the "out of the box" results (too much time consuming).
The only thing to know about the "out of the box" settings of a firewall is that it is never sufficient and does not provides a
good security level, and that it is always weaker than configured with it's highest settings.
The out-of-the-box settings are meant to give a least security level while not blocking the user, but will rarely do more than alerting
you when an application is attempting to access the Internet in a normal way (i.e will not block the leaktests).


Windows 9x/Millenium :

I cease to test on such OS for many reasons.
The first is once again the time needed to do the tests (and triple check them each time), the number of firewalls to test and the number
of the leaktests growing, etc...
Then, the OS core is inherently unsecure and every application can hack and write on the memory of another one without that it
be possible to protect against this, you can't even block a kind of ".vxd" leaktest/rootkit which could do whatever it wants to.
In addition the monitoring OS capabilities are less easy to use (if they simply does not exist) and that's precisely why many known
firewalls and other security products have some of their security feature disabled (DLL monitoring, network driver monitoring, etc...).
Win9x/Me OS can still be less targeted and so seems to be "secure", but I do not advise anyone to rely on these OS to keep confidential information.

** Global LeakTests Ranking **

% of firewall bypassed, 100% means that it bypasses every firewall tested :
Leaktests * Global Score *
LeakTest 20%
Tooleaky 40%
FireHole 30%
Yalta 20%
Outbound - (Win9X/Me)
PCAudit 50%
AWFT 80%
Thermite 90%
CopyCat 100%
MBtest 50%
WallBreaker 100%
PCaudit v2 70%
Ghost 50%
DNStester 80%
Surfer 50%
(current results, will may changed)




** Global Firewall Ranking **

% of leaktest passed by the outbound application filtering component :
Firewall ver(build) * Score *
ZA Pro 5.5.035 beta 70.8%
Kerio 4.1.1 20.8%
OP Pro 2.5(369/369) 75%
Look'n'Stop 2.05p2 79.1%
Norton Personal Firewall 8.0.0.64 29.1%
Sygate Personal Firewall Pro 5.5(2637) 37.5%
Jetico 1.0.21 beta 66.6%
Kasperky Anti-Hacker 1.5.119.0 12.5%
(current results, will may changed)






Tables Legend



result icons :

= the firewall is not vulnerable
= the firewall is vulnerable
= result not available
Application Monitoring capable
Application Monitoring uncapable



points awarded :

one leaktest passed = 1 point
one AWFT point = 1 point
Copycat passed = 2 points



firewall icons :

Zone Alarm Pro
Kerio
Outpost Pro
Look'n'Stop
Norton personal firewall
Sygate Pro
Jetico firewall
Kaspersky AntiHacker
SP1 : XP built-in firewall SP1
SP2 : XP built-in firewall SP2



Click Here to come back to the top of the page